IN THE UNITED STATES K 
In re application of 

Scott A. Morgan et al. 
Serial No.: 09/213,856 
Filed: December 17, 1998 
Title: SPEECH COMMAND INPUT 
RECOGNITION SYSTEM FOR 
INTERACTIVE COMPUTER DISPLAY 
WITH INTERPRETATION OF 
ANCILLARY RELEVANT SPEECH 
QUERY TERMS INTO COMMANDS 

PETITION TO CORRECT INVENTORSHIP UNDER 37 CFR 1.48(a) 

This Petition is to correct the incorrect original naming of 
inventor by adding the following previously unnamed inventor 
of this application: 

Anthony C. C. Temple 

Attached is a Statement Under 37 CFR 1.48(a) .(1) by Anthony 
C.C. Temple. 

Attached is a Verified Statement of Facts by the original 
named inventors. 

Attached is the written Consent of the Assignee. 

Attached is a Declaration by each actual inventor. 

In view of the Statement of Facts by the original named 
inventors, it is submitted that Anthony C. C. Temple was not 
named as an inventor through error without any deceptive 
intention on the part of actual inventors, it is 
respectfully requested that this Petition to add this 
inventor be granted. 

X The Commissioner is hereby authorized to charge payment 
of the following fee associated with this communication or 
credit any overpayment to Deposit Account 09-0447 . A 
duplicate copy of this sheet is enclosed. 
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International Business 

Machines Corporation 
11400 Burnet Road 
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X Please charge my Deposit Account No. 09-0447 in the 



amount of $130.00 . 



Respectfully submitted, 



Leslie Van Leeuwen 
Registration No. 42,196 
Attorney for Applicants 
IPLaw Department 
IBM Corporation 
11400 Burnet Road 
Austin, Texas 78758 
(512) 823-6746 
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: Before the Examiner: 



In re application of 

Scott A. Morgan et al. 
Serial No. : 09/213,856 
Filed: December 17, 1998 
Title: SPEECH COMMAND INPUT 
RECOGNITION SYSTEM FOR 
INTERACTIVE COMPUTER DISPLAY 
WITH INTERPRETATION OF 
ANCILLARY RELEVANT SPEECH 
QUERY TERMS INTO COMMANDS: 



Group Art Unit: 27 41 
Intellectual Property 

Law Department - 4 054 
International Business 

Machines Corporation 
11400 Burnet Road 
Austin, Texas 78758 
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STATEMENT UNDER 37 C.F.R. SECTION 1.48(a)(1) 

Assistant Commissioner of Patents 
Washington, D.C. 20231 

Sir: 

I, Anthony C. C. Temple, state that I am a joint 
inventor of the invention entitled "Speech Command Input 
Recognition System For Interactive Computer Display With 
Interpretation Of Ancillary Relevant Speech Query Terms Into 
Commands" . 

The subject invention was inadvertently filed omitting 
my name as a joint inventor through error and without 
deceptive intent on my part. 

I hereby declare that all statements made herein of my 
own knowledge are true and that all statements made on 
information and belief are believed to be true; and further 
that these statements were made with the knowledge that 
willful false statements and the like so made are punishable 
by fine or imprisonment; or both, under Section 1001 of 
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false statements may jeopardize the validity of the 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
In re application of : Before the Examiner 

Scott A. Morgan et al. 
Serial No. : 09/213, 856 
Filed: December 17, 1998 
Title: SPEECH COMMAND INPUT 
RECOGNITION SYSTEM FOR 
INTERACTIVE COMPUTER DISPLAY 
WITH INTERPRETATION OF 
ANCILLARY RELEVANT SPEECH 
QUERY TERMS INTO COMMANDS 



Group Art Unit: 2741 
Intellectual Property 
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UNDER 37 C.F.R. 1.48(A) » & F1 
r\> g O 

Assistant Commissioner of Patents 

Washington, D. C. 20231 ° 
Sir: 

We, Scott Anthony Morgan, David John Roberts, Craig 
Ardner Swear ingen and Alan Richard Tannenbaum, the inventors 
in the subject application do hereby declare that: 

On November 11, 1997, we submitted to the International 
Business Machines Corporation (IBM) Intellectual Property 
Law Department (IPLaw) at Austin, Texas, Invention 
Disclosure number AT8-97-1230 upon which the present patent 
application is based. The name of Anthony Christopher 
Courtney Temple was included as a coinventor because he had 
always been regarded as a coinventor. The Invention 
Disclosure is attached hereto as Exhibit A. 

Based on information and belief, during the processing 
of the formal papers of the present application, the name of 
Anthony Christopher Courtney Temple was inadvertently 
omitted from the papers . 
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Because Anthony Christopher Courtney Temple spent a 
considerable portion of his time outside the United States, 
it was not unusual for him to review documents and execute 
papers at different times and places from most of the other 
inventors. Accordingly, his absence could be readily 
overlooked on the assumption that he was executing the 
papers elsewhere. 

Based upon information and belief, the omisision of the 
inventor was first discovered in mid-September 1999 when 
papers were being prepared for a counterpart patent 
application in Europe. The matter was brought to the 
attention of IBM IPLaw at Austin, Texas, which investigated 
and commenced the preparation of the present Petition. 

The omission of Anthony Christopher Courtney Temple was 
the result of the above-listed clerical error which went 
unnoticed by the inventors for the reasons set forth above. 

We hereby declare that all statements made herein are 
of our own knowledge and are true and that all statements 
made on information and belief are believed to be true; and 
further that these statements were made with the knowledge 
that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 
1001 of Title 18 of the United States Code and that such 
willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 





Date: 




David John Roberts 
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Title of fnvention (Short & Descriptive) 

Method For Applying Relevance To Speech Recognition Command Systems 



Disclosure No. Functional 1 
AT897-1230 A. Tern 


Manager 


Receiving Date Receiving Time 
11/20/97 14:20:32 


Patent Attorney 
RICHARD A. HENKLER 


Evaluator Evaluation Area 

none 48 


\ Inventor 

Tannenbaurn, Alan R 


Emp. Serial 

272884 


Div./Dept. BIdg. or Zip 

89/tuza 9446 


Location Tel.Number 
Austin 678-4624 


Area Code Electronic Address 
48 TBAUM at AUSVMR 


Manager's Name 
Tony Temple 


Manager's Electronic Address 
TEMPLET at IBMUSM27 



Inventor 

Morgan, Scott A 


Emp. Serial 
409383 


Div./Dept. 
89/AZAS 


BIdg. or Zip 
9446 


Location 
Austin 


Tel.Number 
678-2207 


Area Code 
48 


Electronic Address 
MORGANS at IBMUSM27 


Manager's Name 

Paul Waldo 


Manager's Electronic Address 
PWALDO at IBMUSM20 



Inventor 
Swearingen, Craig A 


Emp. Serial 
409368 


Div./Dept. 
89/AZAS 


BIdg. or Zip 
9446 


Location 
Austin 


Tel.Number 
678-8144 


Area Code 
48 


Electronic ^ddress 
SWEARING at IBMUSM27 


Manager's Name 

Paul Waldo 


Manager's Electronic Address 

PWALDO at IBMUSM20 






Inventor 

Roberts, David J 


Emp. Serial 
077572 


Div./Dept. 

58/4900 


BIdg. or Zip 

osu5 


Location 

Warwick. UK 


Tel.Number 

7664788 


Area Code 
48 


Electronic Address 
ROBERTS at WSDLOF01 


Manager's Name 

David Mackie 


Manager's Electronic Address 
MACKIED at WSDLOF01 








Inventor 

Temple, Tony C 


Emp. Serial 
768748 


Div./Dept. 

89/haha 


BIdg. or Zip 
9446 


Location 
Austin 


Tel.Num.be: 
678-4725 


Area Code 

48 


Electronic Address 

TEMPLET at WSDLOF01 


Manager's Name 

Don Haile 


Manager's Electronic Address 

HAILE at 1BMUSM21 



Table 1. Critical Dates Infornu.tion 


Date invention workable: 


06/25/97 


Used or Planned for product: 


Y 


If so, Product Name? 


Code Name Amethyst 


Release? 




Announce Date? 


03/15/98 


Public Demonstration or Use: 


N 


If so, When? 




Where? 




Disclosed to Non-IBMers: 


N 


If so. When? 




Where? 





CDA in piace? 



Us e in Manufacturing: 
If so, When? 
Where? 



Produci Name? 



, Method For Applyin g Ratavance To Speech Recognition Command Sytwnt Page 2o>S 

. •- . # £7l#>e>lTft (M 

Problem 

There are two main uses for speech recognition systems today: 1) conversion of speech to text, sometimes known as 
dictation, and 2) conversion of speech to commands to control an application or "navigate" through the graphical pres- 
entation of the application by s >eaking the menus, buttons, and names of other visual elements of the interface. 

s 

The problem we are addressing is related to the command and control aspects of speech recognition, and the user's 
ability to choose the correct wcrd or phrase to use in order to accomplish a task. 

So that the user can learn as quickly as possible how to use the speech system to control a desired application, the user 
is able to speak all of the words found in the graphical interface, including the menu bar commands, the buttons within 
dialog boxes, check box labels, radio buttons, etc. Once a user says the name of a menu item, for example, they then 
see a "pull-down" list of sub-m:nu items that they can say. 



For experienced users, they may also say the names of commands in the pull-down sub-menus without actually seeing 
them pulled down. This makes the system efficient for people who do repetitive tasks, and especially for commonly 
used functions such as "print", "save", and "paste", that are normally found in pull-down submenus. 



Because of this feature of enabling words that are not actually visible, there is a potentially large set of words that may 
be spoken at any time. Furthermore, if a user is trying to say the name of a command that is not visible, and it is a 
commonly used command function found in many applications, the user may say the wrong word, since different appli- 
cations use different command names to invoke the same function. An example is "properties", "preferences", and "set- 
tings". 

Current desktop speech control systems attempt to solve this problem by providing a reference which lists all of the 
commands that can currently be spoken (both visible and otherwise). This reference typically takes the form of a 
window on the display screen 'hat lists all of the words broken into categories to help the user deal with the potentially 
large number of command words and phrases. Because the number of entries in this list is large, it has the tendency to 
take a significant amount of screen space, blocking some underlying application interface. Furthermore, because the 
window is frequently used, it tands to stay on the screen, essentially reducing significantly the amount of space that can 
be used for other programs. Finally, the list itself must be navigated so as to find the desired command name. This 
imol'cs scrolling expanding and contracting hierarchical lists, and choosing various category sections. The user must 
locate the desired word by taking their best guess as to where it may be found in the list, and then once found and 
spoken hoping that the command was in fact the correct one for the task at hand. 



Solution 

This invention solves a number of problems found in the existing mechanisms described above. 

tHp mention comnrises an a ternate assistance mechanism that is designed to provide a list of commands that are 

^T^^^^ user is perfor r 8 v^ ia S y '- lh s n describes the lask ' syslem SU88CSts a 

Svely small set of speech commands that could perform the desired task. 

Furthermore, the list is displayed in such a manner as to take up minimal space on the screen, and to be "dismissed" 
once the user has chosen a ward from the hst. 

These two major aspects of the invention are described in more detail below: 

1) RELEVANXE FILTERING 

u „.wMw a user orovides a description of the desired task to be performed, and receives 

^ffiS^^t^^™^ r he ,ask - This ,ist is ,aken rrom thc to,al set 

matds which could possibly be spoken to control an apphcauon. 

The process entails the following steps at the point in time that the user asks for assistance: 



Mtthod For Applying. Belevinct To Spt»ch Recognition Command Systemt 



When first invoked the speech expert displays a small dialog. Two fields are apparent: 1) an entry field with a label 
^t^^^^SSS^iMy^ w«rt to do", and 2) an area which will be filled later with a list of the speech 
commands related to the description, (illustration is available). 

A microohone is displayed and tie user supplies the description. If not recognized, a short message is displayed to ask 
ih ^se? to "peak i aS For example, when used with the IBM Spimply Speaking product the user sees 'Tardon 
me?" bu anSr sysfem might display a more verbose prompt. Once recognized, the entry field displays the 
&^^lhi to l3£ is filled" with a list of application and task related voice commands taken from the active 
vocabulary of the application in its current state. 

If the user sees the command to accomplish the task, the user speaks the word or phrase, it is briefly highlighted to 
acknowledge the recognition, and the application function is invoked. 

If the user determines the list does not contain the correct command, the user either clears the list and starts over with a 
I^H^STSa^tates a new description, which automatically clears the list and displays the new description 
JSgSC^KS commands related to the new description. This modeless interaction is described in 
another disclosure. 

CLAIMS 

Advantages of this invention over known implementations of speech assistance systems: 

1. Filtering by relevance 

2. Automatic expansion or an application's vocabulary with synonyms and variants ofthe commands, us.ng mutil- 
level expansion 

3. Using an expanded vocabulary to drive the speech system, rather than expanding the query terms after the query 
has been made 

4. Using a dialog mechanism rather than a reference window 

5. Allowing for modeless descriptions. The user may just speak another description ifthe first failed to provide the 
desired results. 



Evaluation Questions 

If this problem has been solved before, how was it solved? 

Using reference windows that ;ook up significant screen space and typically floated on top of all other windows, usually 
blocking some application. 

Why Is your solution better? 

Proven, via usability studies, t,at relevance filtering and dialog presentation is faster and has a higher level of satisfac- 
tion. 

Who outside of IBM (competitors) would want to use your solution? 

Desktop speech navigation system vendors (Microsoft. Dragon Systems, Kurzweil, BBN,...) 



Method For Applying Relevanct To Speech Reco gnition Command Systems Page 3 of 5 



A) Determining the dynamic, Active vocabulary". This includes collecting: 

• all words from menus* buttons, and other UI controls, including the invisible, but active words from the currently 
active application window 

• names of macros supplied by the speech system, the application, and the user 

• names of other applications that the user can switch to 

• genekc commands that can be spoken to any application 

• any other words that may be active 

B) Expanding the active vocabulary with related terms. This results in the construction of an "expanded vocabulary", 
wich will be used to control the speech recognition system in the next step. The expansion process employs, but is not 
limited to, the following steps: 

• Each word or phrase in the active vocabulary is added to the expanded vocabulary, with an indication that it is an 
original active vocabulary word or phrase. 

• Each WORD or PHRASE .n the active vocabulary is looked up as an index into a "relevance table". IT found, the 
corresponding contents of the table cell is used to expand the vocabulary. The cell may contain one or many addi- 
tional words or phrases. Each word or phrase added to the expanded vocabulary has an indication of which active 
vocabulary entries caused this expansion. 

• For each PHRASE in the active vocabulary, the phrase is broken into its constituent words, and each word is ilien 
used as an index into the relevance table. Once again, matching table entries are used to expand the vocabulary. 
This continues for all words in a phrase. Each word or phrase added to the expanded vocabulary has an indication 
of which active vocabulary entries caused this expansion. 

• For each PHRASE in the active vocabulary, the phrase is broken into its constituent word-pairs, and each word- 
pair is then used as an index into the relevance table. Once again, matching table entries are used to expand the 
vocabulary. This continues for all word-pairs in a phrase, 

• For some applications, the process continues for n-word sub-phrases. 

• Lists of related words provided by users. 

• Lists of related words provided by ISVs via APIs and other registry mechanisms. 

• For some applications, a synonym dictionary is employed as an additional source of expanded words and phrases 
for words found in the active vocabulary. 

C) Accepting a spoken description of the task at hand, and determining which active vocabulary entries are related. 
This entails waiting for the user to speak a word or phrase. Since the speech recognition system will be controlled by 
only the "expanded vocabulary", any recognized phrase will have been built by the process (B), above, and will therefore 
have an associated indication of the original active vocabulary words or phrases that resulted in the phrase being added 
to the expanded vocabulary. The set of original active vocabulary phrases are collected and presented to the user. 

D) Accenting a spoken command from the presented list of active vocabulary words or phrases from (C). Having 
oresented a list of command words/phrases related to the description of the task at hand, wait for the user to speak one 
of them and then cause the action to occur. Since the speech recognition system is still being controlled by the voeabu- 
larv built in steo (B) above, he user is free to speak either one of the presented commands, or to try describing the task 
in different words So, if a spoken phrase is recognized, but does not match one of the items from step (C), then step 
(q I is repeated for this new description. This enables a "modeless" interaction which is a significant advance, and hence 
described in a separate disclosure. 

2) INTERACTION 

This is the aspeci of the invention that deals with the interaction between the user and the presentation mechanism. 

This invention is intended to be employed whenever a user has either exhausted other means of assitance to determine 
u ^ to oerform a task, or the user prefers to use this invention because for example, the 

only has a concept idea of how to describe the task; the menu commands are 

not yet familiar. 

• ,h; a a «kiance scheme by saying "Speech Expert", or "What Do I Say?" or some other trigger phrase, 

^^^gVSS^ft^oL^ for different users. For the purposes of this disclosure, wc will call u 

"Speech Expert". 
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How could IBM discover that competitors were using your solution^ )&( ( & ^ ^ 

Obvious in the user interface, and system and developer toolkit descriptions. 
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IN THE UNITED STATES PATENT 
In re application of : 

Scott A. Morgan et al. 
Serial No.: 09/213,856 
Filed: December 17, 1998 
Title: SPEECH COMMAND INPUT 
RECOGNITION SYSTEM FOR 
INTERACTIVE COMPUTER DISPLAY 
WITH INTERPRETATION OF 
ANCILLARY RELEVANT SPEECH 
QUERY TERMS INTO COMMANDS 



AND TRADEMARK OFFICE 
Before the Examiner: 

Group Art Unit: 2741 
Intellectual Property 

Law Department - 4054 
International Business 

Machines Corporation 
11400 Burnet Road 
Austin , Texas 78758 
Date : , 



CONSENT TO ADD INVENTOR 
Assistant Commissioner of Patents 
Washington , D . C . 20231 
Sir: 
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International Business Machines Corporation, theo 

o 

original sole assignee of the entire right, title and 
interest in and to the above-identified application hereby 
consents to the addition of Anthony Christopher Courtney 
Temple as joint inventor in the U.S. Patent Application 
identified above. 
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As a below named inventor, I hereby declare that: 



DECLARATION AND POWER OF ATTORNEY FOR 



PATENT APPLICATION 




My residence, post office address and citizenship are as stated below next 
to my name; 

I believe I am the original, first and sole inventor (if only one name is 
listed below) or an original, first and joint inventor (if plural names are 
listed below) of the subject matter which is claimed and for which a patent is 
sought on the invention entitled 

SPEECH COMMAND INPUT RECOGNITION SYSTEM FOR INTERACTIVE COMPUTER DISPLAY WITH 
INTERPRETATION OF ANCILLARY RELEVANT SPEECH QUERY TERMS INTO COMMANDS 

the specification of which (check one) 

is attached hereto. 

X_ was filed on December 17. 1998 

as Application Serial No. 09/213.856 

and was amended on 



I hereby state that I have reviewed and understand the contents of the above 
identified specification, including the claims, as amended by any amendment 
referred to above. 

I acknowledge the duty to disclose information which is material to the 
patentability of this application in accordance with Title 37, Code of Federal 
Regulations, §1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, §119 
of any foreign application(s) for patent or inventor's certificate listed below 
and have also identified below any foreign application for patent or inventor's 
certificate having a filing date before that of the application on which 
priority is claimed: 

Prior Foreign Application(s) : Priority Claimed 



I hereby claim the benefit under Title 35, United States Code, §120 of any United 
States application(s) listed below and, insofar as the subject matter of each of 
the claims of this application is not disclosed in the prior United States 
application in the manner provided by the first paragraph of Title 35, United 
States Code, §112, I acknowledge the duty to disclose information material to 
the patentability of this application as defined in Title 37, Code of Federal 
Regulations, §1.56 which occurred between the filing date of the prior 



(if applicable) 



Yes 



No 



(Number) 



(Country) 



(Day/Month/Year) 
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DOCKET NUMBER: AT9-98-343 

application and the national or PCT international filing date of this 
application: 



(Application Serial #) (Filing Date) (Status) 

I hereby declare that all statements made herein of my own knowledge are true and 
that all statements made on information and belief are believed to be true; and 
further that these statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or imprisonment, or both, 
under Section 1001 of Title 18 of the United States Code and that such willful 
false statements may jeopardize the validity of the application or any patent 
issued thereon. 

POWER OF ATTORNEY: As a named inventor, I hereby appoint the following attorneys 
and/or agents to prosecute this application and transact all business in the 
Patent and Trademark Office connected therewith. 

John W. Henderson, Jr., Reg. No. 26,907; Thomas E. Tyson, Reg. No. 28,543; Robert 
M. Carwell, Reg. No. 28,499; Jeffrey S. LaBaw, Reg. No. 31,633; Douglas H. 
Lefeve, Reg. No. 26,193; Casimer K. Salys, Reg. No. 28,900; David A. Mims, Reg. 
No. 32,708; Richard A. Henkler, Reg. No. 39,220; Volel Emile, Reg. No. 39,969; 
James H. Barksdale, Jr. Reg. No. 24,091; Anthony V. England, Reg. No. 35,129; 
Christopher A. Hughes, Reg. No. 26,914; Edward A. Pennington, Reg. No. 32,588; 
John E. Hdel, Reg. No. 26,279; Joseph C. Redmond, Jr., Reg. No. 18,753; Leslie 
A. Van Leeuwen, Reg. No. 42,196; Marilyn S. Dawkins, Reg. No. 31,140; and J. B. 
Kraft, Reg. No. 19,226. 



Send correspondence to: Leslie Van Leeuwen, International Business Machines 
Corporation, Intellectual Property Law Department, Internal Zip 4054, 11400 
Burnet Road, Austin, Texas 78758 and direct all telephone calls to Leslie Van 
Leeuwen, (512) 823-6746. 



FULL NAME OF SOLE OR F I RS;? INVENTOR: 

INVENTORS SIGNATURE: 

RESIDENCE : 3609 Kentf ield Road 
Austin, Texas 78759 




Scott Anthony Morgan 



CITIZENSHIP: United States 



POST OFFICE ADDRESS: 3609 Kentf ield Road 

Austin. Texas 78759 
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DOCKET NUMBER: AT9-98-343 

FULL NAME OF SECOND INVENTOR: David John Roberts 

INVENTORS SIGNATURE Tp'^l jrtU* DATE: ^o- T^tZfeT^gt^ 131 3 

RESIDENCE: Sycamore Lodge. Church Street 

Stockton. Warwickshire CV23 8J6 
United Kingdom 

CITIZENSHIP: United Kingdom 

POST OFFICE ADDRESS: Sycamore Lodge. Church Street 

Stockton. Warwickshire CV23 8JG 
United Kingdom 



FULL NAME OF THIRD INVENTOR: Craio Ardner Swear ingen 
INVENTORS SIGNATURE: 



INVENTOR: Craig Ardner Swear ingen . 



RESIDENCE: 8905 Martha 'st/Drive 
Austin. Texas 78717 

CITIZENSHIP: United States 

POST OFFICE ADDRESS: 8905 Martha's Drive 

Austin. Texas 78717 




FULL NAME OF FOURTH INVENTOR; Alan^Ri chard Tannenbaum 

INVENTORS SIGNATURE: 

RESIDENCE: 3801 Grevstone Drive 
Austin. Texas 78731 



■ Kicnara lannenpaum / 



CITIZENSHIP: United States 

POST OFFICE ADDRESS: 3801 Grevstone Drive 

Austin. Texas 78731 



FULL NAME OF SOLE OR FIFJJ^VTlNTOrI Anthony Christopher Courtney Temple 
INVENTORS SIGNATURE S \ DATE: 2^ Sete+dec 




RESIDENCE: "Ferndene^rTittle Back Lane 

Hellidon. Northants. United Kingdom NN11 6GE 

CITIZENSHIP: United Kingdom 

POST OFFICE ADDRESS: "Ferndene". Little Back Lane 

Hellidon. Northants. United Kingdom NN11 6GE 
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Applicants: 
Application No.: 
Filed: 
For: 



< I I'TIMi ^ TTNDF R 37 C.F.R. SECTION 
Scott A. Morgan et al. 
09/213 ,856 
December 17, 1998 
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International Business Machines Corporation , a Corporation, certifies that 
it is the assignee of the entire right, title and interest in the patent 
application identified above by virtue of either: 

A. X An assignment from the inventor (s) of the patent application 
identified above. The assignment was recorded in the Patent and Trademark 
Office at Reel 9668, Frame 0432, or which a copy thereof is attached. 

The undersigned (whose title is supplied below) is empowered to act on 
behalf of the assignee. 

I hereby declare that all statements made herein are of my own knowledge 
and are true, and that all statements made on information and belief are 
believed to be true; and further, that these statements are made with the 
knowledge that willful false statements, and the like so made, are 
punishable by fine or imprisonment, or both, under Section 1001, Title 18 
of the United States Code, and that such willful false statements may 
jeopardize the validity of the application or any patent issuing thereon. 

Date: TUciry aw 

Name : Jeffrey L. Forman 

Title: Manager , Information Services 
Intellectual Property Law 
Washington 



Signature: 
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LETTER OF AUTHORITY 



., ailt intellectual Property and 
* H.r«h.ii c. Pheip.. corporation (M. • 

f Uo».i»9 of Xnt.xn.tlon.1 e..in.,, **1 ta . ppre v. 

w d e*ecuta documents on WiHI 01 «» office* aervicing 

the Patent. * ^"ration or ™<f\™ Mt6 , ppB al 

any country or region of th. ioMI p0W ar B of 

_ . 1-,-iuiiino. but not llMltea to. v**+ 
pM c..din 9 .. inching. * v « ifleatloni no.i5.tion, of 

for the registration of patents a, available for licena.n,. rep 

of invention, and petition, for waiver of patent to any 

department or agency of the United State, Government, and . 

...ign-ent.. ii«n«« and other in.tnu.ent. °f ^"^J 

covenant right, in patent, and invention,, to Jeffrey 

Manager, Inf.r-.tion Service.. Int.ileetu.l Property Law. 
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Vice President - Intellectual Property & Licensing 
International Business Machines Corporation 
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1. Name of conveying party (ies): 



Scott A. Morgan 
David J. Roberts 
Craig A. Swearingen 
Alan R. Tannenbaum 



2. Name and address of 
receiving party (ies) 

International Business 
Machihes Corporation 
New Orchard Road 
Armonk, New York 10504 



3. Nature of conveyance: 

X Assignment 
Execution Date: / £t-/<r-9 9r 



4. Application number (s) or patent number (s): 

If this document is being filed together with a new application, the 
execution date of the application is /3 — / T-^ tr -f K 



A. Patent Application No.(s) 



B • Patent No . ( s ) 



5. Name and address of party to whom 
correspondence concerning document 
should be mailed: 

Richard A. Henkler 
IBM Corporation 
Intellectual Property Law 
Internal Zip 4054 
11400 Burnet Road 
Austin, TX 78758 
(512) 823-0962 



6. Total number of applications 
and patents involved: 1 



7. Total fee (37 CFR 3.41) $ 40.00^ 

X Authorized to be charged to 
deposit account 

8. Deposit Account Number: 09-0447 



i2/£5/ mS EMiRUB — OWOffiZfi fr??M7 — M213C56 



40.00 CH 
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To the best of my knowledge and belief, the foregoing information is true 

and correct and any attached r^jopy is a true copy o£ the original document. 

Name of Person Signing Signature Date 
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Whereas, we, 



ASSIGNMENT 



(1) Scott A. Morgan 
County of Travis 

and 

(2) David J. Roberts 
Warwickshire 

and 

(3) Craig A. Swearinqen 
County of Williamson 

and 

(4) Alan R. Tannenbaum 
County of Travis 



of Austin T 

and State of Texas 



of Stockton . 

in the United Kingdom . 



of Austin . 

and State of Texas . 



of Austin . 

and State of Texas . 



have invented certain improvements in 

SPEECH COMMAND INPUT RECOGNITION SYSTEM FOR INTERACTIVE COMPUTER DISPLAY WITH 
INTERPRETATION OF ANCILLARY RELEVANT SPEECH QUERY TERMS INTO COMMANDS 

and executed, respectively, a United States patent application therefor on 



tes p 

A 3 // 



= , 192&, and 

= , 192&, and (4) 



(Dates Inventors Signed Declaration) (1) 
V) - and (3) 

. 7998. 

Whereas, INTERNATIONAL BUSINESS MACHINES CORPORATION, a corporation 
of New York, having a place of business at Armonk, New York 10504, (hereinafter called 
IBM), desires to acquire the entire right, title and interest in the said application and 
invention, and to any United States and foreign patents to be obtained therefor; 

Now therefore, for a valuable consideration, receipt whereof is hereby acknowledged, 
we, the above named, hereby sell, assign, and transfer to IBM, its successors and assigns, 
the entire right, title and interest in the said application and invention therein disclosed for 
the United States and foreign countries, and all rights of priority resulting from the filing of 
said United States application, and we request the Commissioner of Patents to issue any 
Letters Patent granted upon the invention set forth in said application to IBM, its successors 
and assigns; and we hereby agree that IBM may apply for foreign Letters Patent on said 
invention and we wilt execute all papers necessary in connection with the United States and 
foreign applications when called upon to do so by IBM. 

AtAS73=xs on / ^ //& 



(1) Signed at /rt4ST3=xs on 



im 




Scott A. Morgan 
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and (2) Signed at == _ =a= on ====== = , 



Itow J 7. Roberts 



and (3) Signed at (A on 




Craig A. Swearingen 
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Whereas, we, 



ASSIGNMENT 



(1) SCfftt A. M,prqa.n of Austin , 

County of Jxavi5__ and state of T^as., 

and 

(2J David J. Roberts of Stockton . 

&E«Mshj£e_ in the United KinaHnm 

and 

J 3 ) -feajq A. Swearinqen 0 f Austin . 

county of Williamson and state of T€xas 

and 

(4) AUq R. Tannenbaum 0 f Austin . 

County of .Travis and State of Texas r 

have invented certain improvements in 

SPE ECH COMMAND INPUT R ECOGNITION SYSTFM FOR TNTFRAr TTur COMPUTFR nTSPI AV urru 
INTERPRFTA HON OF ANClT. arv" VA AJht spffch mSff ^ E InTO m ^ 

and executed, respectively, a United States patent application therefor on 

(Dates Inventors Signed Declaration) (1) /aw and 0 

(2) S^WiS , i m , and flJ — ^ r4; 

— '^28- 

Hfterea* INTERNATIONAL BUSINESS MACHINES CORPORATION, a corporation 

HS^l ' 8 ° Phce o/W:OT ^ ^""^ ^ w H»* ^5(W, (hereinafter called 
IBM), desires to acquire the entire right, title and interest in the said application and 
invention, and to any United States and foreign parents to be obtained therefor; 

Now therefore, for a valuable consideration, receipt whereof is hereby acknowledged 
we. the above named, hereby sell, assign, and transfer to IBM. its successors and assigns' 
the enure right, title and interest in the said application and invention therein disclosed for 
the United States and foreign countries, and all rights of priority resulting from*®* filing of 
said United States application, and we request the Commissioner of Patents to issue any 
Utters Patent granted upon the invention set forth in said application to IBM. its successors 
?™Jf'* W \, w agree that IBM may apply far foreign Letters Patent on said 

invention and we will execute all papers necessary in connection with the United States and 
foreign applications when called upon to do so by IBM. 

(I) Signed at . on . 19S&. 



Scott A. Morgan 
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and (2) Signed ai VWuii-cte on D^ccu^W 1.5 




David J. Roberts 



and (3) Signed at Q n 



and (4) Signed at === ^^^^ on 



Craig A. Swearingen 



Alan IL Tannenbawn 
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DOCKET NUMBER: AT9-9B-343 
DECLARATION AND POWER OF ATTORNEY FOR 
/ PATENT APPLICATION 

As a below named inventor, I hereby declare that: 

to my Ler 1denCe ' P0St ° ffiC6 addr6SS and citUenship arB as stated *elow next 

4ff?!!^ C0 - M ^ D * NPUT ^^CW MITJON SYSTEM FOR INTFRArTTVF rn MPirrrp DISPLAY WITH 
OTRPRETAtYqn OU tfjaj^^ QUERY 7 1- R MS TMTn rn^ ft[ftMpg HT ™ 

the specification of which (check one) 

Xjjis attached hereto. 

> 

was filed on 

as Application Serial No. 



and was amended on 



(if applicable) 

LISTi? i tate ^ at 1 have rev1ewed and understand the contents of the above 
iJESdlo a P bo C ve. iCati0n ' inClud1n9 thG Cla1ms ' as amended 5 «5 SnSSt 

LtPntTn 6 -? 6 i h .l. duty , t0 discl «e information which is material to the 
Re ula??ons S6 h7S app1iCat10n in accordance with Title 37, Code of Federal 



iSf h «t b ^ C J airo for « f 9 n Parity benefits under Title 35, United States Code 5119 
of any fore ig n application(s) for patent or inventor's certificate listed below 
£?t fiwte^h^ 1 '; 6 ?-?^ 0 "^ f £ rB > a PP^"tion for patent or nventoJ's 
lr?olnTu cSed: 3 fl1 '" 9 ^ that ° f the ^""tion °" which 

Prior Foreign Application^): p riori - ty Claimed 

( Number > (C^ntrT) (Day/Month/Year ) — Y6S — N ° 

Stltlf JiiVr'^ 1 " ?W* . under Title 35 ' United st * tes c ° d e> §120 of any United 
"J claims o ^XiiHn v Ud ^ el0 '? a " d ' inS0far as the subjectmatter ofVach S 
aD^icATnn ^It applicat 10 n is not disclosed in the prior United States 

the patentability «f th^lln 9 - 6 ?. e duty }° dlscl °se information material to 
Regulations si f« i-'t applicat 2°" as defined ™ Titl e 37, Code of Federal 
aeguiations, §1.56 which occurred between the filing date of the prior 
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( DOCKET NUMBER: AT9-98-343_ 
atii«?io 0 S: the natl *° nal " PCT Rational fi ling date of this 



\ 



(Application Serial #) (Filing Date) ^ (status) 

H«^nts L ,h S . e 1 S i h ,ement 5 " ere ttittl 1,16 knotledje thai : wliwlpf,"? 

patent and Trademark Office connected therewith anSact 91 ' business "> the 

fcj?: srs: ^b N 6 °- 42 ' 196; Mariiyn s - oaiAi » s - Re ' "° «.i& L o s, B e 

S™ 1 ^ W S^* t °=/^ h ^*\ Ha * , ' r ' Internal,-.^ e«i M ss Machines 

FULL NAME OF SOLE OR FIRST INVENTOR: Scott Anthnnv Mn^n 
INVENTORS SIGNATURE: DATE . 



RESIDENCE: 3&Q9 Kentfield Rn a rf 
Austin. l* m 73750 

CITIZENSHIP: United Stat**. 

POST OFFICE ADDRESS: 3§ fl9_ICentf ield Rnari 

AMstin^ Texas 
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FULL NAME OF SECOND INVENTOR: David John Rnher jt 

INVENTORS SIGNATURe T^uvIX^A^ ^K-S^b cflATI 

RESIDENCE: Sycamore Lo dge. Church Strep t 
S.t.ockton. Warwickshire cyp? ftffi 
United King dom 

CITIZENSHIP: United Kingdom 

POST OFFICE ADDRESS: Sycamore 1 adge. Church Street 

Stockton. Warwickshire CV23 BJfr 
United Kingd om 



£fo c*j ft- 'if*; 



of 1y one naae to 
plural panes ero 
■hSch a patent is 



^ FULL NAME OF THIRD INVENTOR: Craio Ardner Swearinoen 
INVENTORS SIGNATURE: DATE: 



RESIDENCE: 8905 Martha's Drive 
Austin. Texas 78717 

CITIZENSHIP: United States 

POST OFFICE ADDRESS: 890S Martha's nriwo 

Austin. Texas 78717 



FULL NAME OF FOURTH INVENTOR: Alan Richard T i mn»nh Mff 

INVENTORS SIGNATURE: DATE: 

RESIDENCE: 3801 Grevstone Drivo 
Austin. Texas 78731 

CITIZENSHIP: United States 



POST OFFICE ADDRESS: 3801 Grevstone Drive 

Austin. Texas 7873 1 
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